Modern data centers are comprised of magnetic hard disk drives (HDD) and/or SSDs. Although SSDs operate using different technology than HDDs, SSDs provide the same interface and functionality as HDDs for compatibility with existing data center operating systems. A typical SSD is comprised of non-volatile NAND media (comprised of memory blocks and pages capable of retaining data when power is disconnected), a controller and an interface (e.g., PCIe, SAS, SATA, or any other interface).
NAND SSDs are equipped with a flash translation layer (FTL) in the SSD controller that translates logical block addresses from a host device (e.g., addresses used in read, write commands from a computer) to low-level flash operations for the associated physical block and page addresses of the NAND SSD. The embedded FTL also performs numerous other functions, including: error code correction (i.e., use of redundant data, or parity data to recover a message with errors); garbage collection (i.e., identifying/invalidating stale data in blocks of memory for deletion to free space for future writes); scheduling (i.e., controlling the order in which read and write commands from the host device are executed); over-provisioning (i.e., SSD memory reserved for maintaining write speed); and wear-leveling (i.e., spreading writes across the blocks of the SSD as evenly as possible to ensure that all blocks of the device wear at a roughly similar rate). Because the FTL of NAND SSDs can accommodate the logical interface of host devices, NAND SSDs easily integrate with standard hard disk drive (HDD) compatible interfaces and offload overhead that would otherwise be needed for the host device to perform the functions of the FTL.
Although traditional NAND SSDs are easy to integrate and have the advantage of the FTL offloading host device overhead, a traditional host device HDD interface is not capable of taking full advantage of the performance of a NAND SSD. For example, the host device is not capable of issuing low-level commands that govern how data is programmed and erased in the SSD and the SSD is not aware of when the host device will issue read and write commands. Accordingly, while both the FTL and the host device employ best effort approaches for performing their independent functions, the result is still inefficient utilization of NAND SSD resources, including unpredictable data storage and increased wear-leveling.
One technique for increasing efficiency between a host-device with a conventional HDD-interface and the FTL of a NAND SSD is through an application program interface (API) that shares valuable information between such devices (i.e., through Storage Intelligence). However, this has not proven to be an achievable approach as the complexity of coordinating FTL functionality with host device functionality can further reduce efficiency of the NAND SSD.
Another technique involves using a NAND SSD that does not have a firmware FTL, which requires the host device operating system to manage the physical solid-state storage. The Linux 4.4 kernel is an example of an operating system kernel that supports open-channel SSDs which follow the NVM Express specification, by providing an abstraction layer called LightNVM. However, open-channel SSDs require significant overhead as the host device operating system must manage the functionality of each open-channel SSD.
For example, a modern data center (i.e., a system) stores and process petabytes of data on a monthly basis. This storage includes a physical layer of disks (e.g., open-channel SSDs) and software abstraction that exposes the disks to applications that control physical storage within such disks. Separating the software abstraction layer from the physical layer for each open-channel SSD has the disadvantage of consuming an increasingly larger proportion of system resources as additional open-channel SSDs are deployed within the system.
Accordingly, there is an unmet demand for a system that can easily scale to accommodate any number of open-channel SSDs and can efficiently manage a plurality of open-channel SSDs in groups or pools rather than at the individual device level, thereby reducing system overhead, increasing the efficiency of garbage collection and reducing wear on individual open-channel SSDs.